1. Field of the Invention
The present invention relates to a method and apparatus for exemplar-based inpainting in a multi-scale space using a Laplacian pyramid, among inpainting methods for recovering a damaged portion or a portion to be removed of an image so that the recovered portion naturally matches with a remaining portion of the image.
2. Description of the Related Art
Inpainting refers to a method for recovering a damaged portion or a portion to be removed of an image so that the recovered portion naturally matches with a remaining portion of the image. Also, the inpainting may be used to remove even a normal portion or object when the normal portion or object is desired to be removed. An exemplar-based inpainting method out of various inpainting methods extracts part of a known region of an image and fills a region to be recovered with the extracted part.
Generally, an exemplar-based inpainting algorithm may iteratively perform recovery of a predetermined patch region and reconstruct an entire recovery region. Since the exemplar-based inpainting algorithm recovers a region having structural features when iterating the recovery process, plausibility may be maximized. To maximize the plausibility, the exemplar-based inpainting algorithm determines a process of selecting a recovery region to be recovered and a process of selecting a patch to fill the selected recovery region. Recovery of one cycle of the exemplar-based inpainting algorithm may be performed through the aforementioned two processes.
A conventional inpainting method used to focus on the recovery region selecting process and the patch selecting process to maximize the plausibility.
In addition, conventionally, inpainting methods use factors such as brightness, gradient, contour, a Hessian Matrix, Belief propagation, sparsity and the like. The conventional inpainting methods extract features of the factors and combine the extracted features so that structural features and texture features are naturally connected.
Therefore, the overall recovery process focuses on the recovery region selecting process and the patch selecting process to select a patch to match the recovery region best. Furthermore, the conventional inpainting methods directly use resolution of an input image.
In an image region, usually, the structural features may be distributed in a low frequency region while the texture features are distributed in a high frequency region. Therefore, when the resolution of the image is relatively low, the image is blurred and the structural features may remain over an entire screen. As the resolution is higher, the texture features are added and the image becomes clear.
The inpainting method using a multi-scale space may maintain the structural features while preserving the texture. According to conventional inpainting methods using the multi-scale space, an inpainting method fixed to one resolution divides a recovery region into several regions and process independently at proper resolution or mix patches selected from several resolutions. According to the conventional methods, it is difficult to apply the inpainting method using the multi-scale space in direct inpainting.
The exemplar-based inpainting method may not be able to obtain a natural recovered image when the structural features and the texture features are not considered together in the recovery region. When the structural features do not properly match, contents of the image may be inconsistent. When the texture features do not properly match, a boundary line of the recovery region is noticeable and therefore naturalness of the image may be reduced. In an image, the structural features are distributed usually in the low frequency region while the texture features are distributed usually in the high frequency region. Accordingly, there is a desire for a method to merge the two regions.